The Total Least Squares Problem with Multiple Right-Hand Sides

The total least squares (TLS) techniques, also called orthogonal regression and errors-in-variables modeling, see [15, 16], have been developed independently in several areas. For a given linear (orthogonally invariant) approximation problem AX ≈ B, where A ∈ Rm×n, B ∈ Rm×d, X ∈ Rn×d, the TLS formulation aims at a solution of a modified problem (A + E)X = B + G such that min ‖[G,E]‖F . The algebraic TLS formulation has been investigated for decades, see the early works [5], [4, Section 6], [13]. In [6] it is shown that even with d = 1 (which represents a problem with the single right-hand side Ax ≈ b, where b is an m-vector) the TLS problem may not have a solution and when the solution exists, it may not be unique. The classical book [14] introduces the generic-nongeneric terminology representing a commonly used classification of TLS problems. If d = 1, then the generic problems simply represent problems that have a (possibly nonunique) solution, whereas nongeneric problems do not have a solution. This is no longer true for d > 1. For d > 1, [14] analyzes only two representative cases characterized by the special distribution of singular values of the extended matrix [B,A]. A general case is not analyzed—it is considered only as a perturbation of one of the special cases. The so called classical TLS algorithm given in [14] computes some output X for any data A, B, but the relationship of the output X to the original problem is, however, not clear. The single right-hand side problem has been recently revisited in a series of papers [9, 10, 11]. Here it is shown that the problem does not have a solution when the collinearities among columns of A are stronger than the collinearities between R(A) and b. An analogous situation may occur for d > 1, but here different columns of B may be correlated with different subsets of columns of A. Therefore it is no longer possible to stay with the generic-nongeneric classification of TLS problems. This is also the reason why the question remained open in [14]. In the first part of our contribution we try to fill this gap and investigate existence and uniqueness of the solution of the TLS problem with d > 1 in full generality. We suggest a classification of TLS problems revisiting and refining the basic generic-nongeneric terminology, see [12, 7]. A core reduction concept introduced in [11] makes a clear link between the original data and the output of the classical TLS algorithm for the problems with the single righ-hand side (see also [7]). Therefore the core reduction is an appropriate tool for understanding the TLS problem with d = 1. In the second part of this contribution we introduce an extension of the core reduction for multiple right-hand sides problems. Following [11], we employ the SVD of A which allows us to define the core problem and show its fundamental properties. Then we show how the core problem can be obtained by the band generalization of the Golub-Kahan iterative bidiagonalization algorithm (also called band-Lanczos algorithm) proposed for this purpose by Åke Björck in a series of lectures [1, 2, 3]. We show, together with other results in progress (see [12, 8]), that both approaches (based on the SVD of A and on the band algorithm) give the same core problem up to an orthogonal transformation. Using the core reduction, we illustrate some particular difficulties which are present in the TLS problems with d > 1.